typal

Typal

typal Keeps Types In XML files And Converts Them To (1) JavaScript JSDoc, (2) Closure Externs and (3) Markdown Documentation. It is the alternative to TypeScript definitions and utilises the power of JSDoc for excellent developer experience, documentation readability and compiler annotation. It also makes integration between Google Closure Compiler and VSCode easy, using the JSDoc notations that are understood by both at the same time.

The package's main use is as the CLI tool to generate typedefs, but it also has an API for parsing types and properties with regular expressions.

yarn add -D typal
npm i typal --save-dev

Table Of Contents

• Table Of Contents
• Purpose
• CLI
  • Standard
  • Closure
  • Externs
  • Typal Arguments
  • Missing Types Warnings
  • Keeping Types In Separate File
    • User Snippet
  • Migration
• Schema
• Markdown Documentation
  • Type
  • Example
• Copyright & License

Purpose

The main purpose of this package is to generate JSDoc annotations that are understood both by VSCode, and compatible with Google Closure Compiler via its externs system. The project deliberately deviates from TypeScript and is meant for JavaScript development, and it proves that typing can be achieved perfectly well with JSDoc. It's idea is to store files in an XML file, and then embed them in JS and README files and externs.

The solutions provided by Typal are:

1. Manage types from an external XML location.
2. Compile types into JSDoc compatible both with GCC and VSCode.
3. Compile types as externs for GCC and use in other packages.
4. Place types' descriptions as formatted tables in markdown (used in Documentary).
5. Improve the DevX by increasing the visibility of functions' APIs.

To tune in, start with the Wiki Page:

🥇🥈🥉Naïve, JSDoc And Closure Use Cases For Typedefs

CLI

Typal is the command-line utility that is used to manage JSDoc types in JavaScript source files. The typedefs are now sourced from the types.xml file and embedded on demand. There are 3 modes to embedding types:

1. Standard, no flags required: places only VSCode compatible code. Can be used when no Closure-compilation will be performed on packages. Does not utilise namespaces. Expands the parameters of complex types for better visibility.

   Show Standard JSDoc

   /** * @param {Conf} conf The configuration object. * @param {string} conf.source The source of where to read the data. * @param {boolean} [conf.closeOnFinish=true] Closes the stream when done. Default `true`. * @param {TransformOptions} options */ const prog = (conf, options) => {} /* typal example/cli/types.xml */ /** * @typedef {import('stream').TransformOptions} TransformOptions * @typedef {Object} Conf The configuration object. * @prop {string} source The source of where to read the data. * @prop {boolean} [closeOnFinish=true] Closes the stream when done. Default `true`. */

2. Closure with -c flag: suppresses standard typedefs' annotations so that Closure Compiler does not show warnings. Introduces namespaces for internal as well as external APIs to make types' sources more visible.

   Show Closure JSDoc

   /** * @param {_typal.Conf} conf The configuration object. * @param {string} conf.source The source of where to read the data. * @param {boolean} [conf.closeOnFinish=true] Closes the stream when done. Default `true`. * @param {stream.TransformOptions} options */ const prog = (conf, options) => {} /* typal example/cli/types.xml */ /** * @suppress {nonStandardJsDocs} * @typedef {_typal.Conf} Conf The configuration object. */ /** * @suppress {nonStandardJsDocs} * @typedef {Object} _typal.Conf The configuration object. * @prop {string} source The source of where to read the data. * @prop {boolean} [closeOnFinish=true] Closes the stream when done. Default `true`. */ /** * @suppress {nonStandardJsDocs} * @typedef {import('stream').TransformOptions} stream.TransformOptions */

3. Externs with -e flag: generates types only understood by the Google Closure Compiler, primarily in the externs.js file. These types do not have any meaning for the coding process and are only used in compilation either as types for programs, or externs for libraries.

   Show Externs JSDoc

   /* typal example/cli/types.xml */ /** @const */ var _typal = {} /** * The configuration object. * @typedef {{ source: string, closeOnFinish: (boolean|undefined) }} */ _typal.Conf

Typal Arguments

$ typal source [--closure|externs] [-o output] [-vh]

The following arguments are supported by this software.

Argument	Short	Description
source		The path to the source file or directory with files to embed types into. Can specify multiple values, e.g., typal types/index.js types/vendor.js.
--output	-o	The destination where to save output. If not passed, the file will be overwritten. If - is passed, prints to stdout.
--closure	-c	Whether to generate types in Closure mode.
--useNamespace	-u	Generate JSDoc for functions using namespaces.
--externs	-e	Whether to generate externs for GCC.
--types	-t	Comma-separated location of files to read types from.
--template	-T	Scans the input file for @type comment in functions' JSDoc, and inserts the annotations from types' files.
--migrate	-m	Extracts types from JavaScript source code and saves them into the types.xml file specified in the output option.
--help	-h	Print the help information and exit.
--version	-v	Show the version's number and exit.

Typal will look for its marker in the source files, and insert all types definitions below it. There must be a single new line after the marker, even at the bottom of the file. It is possible to override the arguments, or pass them via the marker itself. When these are specified, there is no need to supply them via the CLI.

function sourceCode() {}

/* typal types/index.xml [closure|externs] [skipNsDecl] [noSuppress] [ignore:_ns.Type,Type] */
_ // remember new line!

• closure: enable the closure mode;
• externs: enable the externs mode;
• noSuppress: don't add @suppress annotations (see the files section below).
• ignore:_nsType,Type: the types to ignore when placing JSDoc into JS files. This can be useful, for example, when the package is built with Depack and has no dependencies, but depends on imported types from other packages. Therefore, these imported types need to be vendored using a separate file, and then imported from there, rather than from their original source file. See @zoroaster/mask/types/vendor.js and @zoroaster/mask/types/index.js for a practical application.
• skipNsDecl: Disables the declaration of the namespace. The types will still be prefixed with a namespace, but it won't be declared at the top as /** @const */ var ns = {}. This is useful when the externs are split by multiple files, and the namespace will only need to appear in one of them, otherwise the Variable _ns declared more than once. error will be thrown.

Missing Types Warnings

When placing JSDoc into source code files where functions are annotated with @params, Typal in addition to expanding object arguments into the developer-friendly notation as discussed above, will check to see if the types were found in the xml files specified in via the /* typal types.xml */ marker to warn of possible errors. This feature aims at helping to identify when some annotations were not done properly, e.g., when missing a namespace, an import, or when type names become outdated. This does not work for record types such as {} since although we have a parser for types themselves, we only use a regular expression which cannot understand things like @param {{ s: string, t }} at the moment. Also only Closure-style types are parsed, i.e., VSCode JSDoc is not supported right now, and the union must be explicitly put in parenthesis.

/**
 * @param {stream.Writable} writable
 * @param {stream.Readable} readable
 * @param {_ns.Type} type
 * @param {_ns.MissingType} missingType
 * @param {Array<_ns.MissingType>} array
 * @param {Promise<MissingType>} promise
 * @param {Object<string, _ns.MissingType>} object
 * @param {(Type | MissingType | _ns.Type)} union
 * @param {(s: string) => number} error
 * @param {MissingType & Type2} intersection Only first type will be parsed
 * @param {string} string
 */
function example (
  writable, readable,
  type, missingType,
  array, promise, object, union,
  error,
  string,
) {}

/* typal example/warnings.xml */

Detected type marker: example/warnings.xml
Type stream.Readable was not found.
example/warnings.js:3:11
Type _ns.MissingType was not found.
example/warnings.js:5:11
Type _ns.MissingType in Array<_ns.MissingType> was not found.
example/warnings.js:6:11
Type MissingType in Promise<MissingType> was not found.
example/warnings.js:7:11
Type _ns.MissingType in Object<string, _ns.MissingType> was not found.
example/warnings.js:8:11
Type Type in (Type | MissingType | _ns.Type) was not found.
example/warnings.js:9:11
Type MissingType in (Type | MissingType | _ns.Type) was not found.
example/warnings.js:9:11
Error while parsing the type (s: string) => number
Expecting closing )
example/warnings.js:10:11
Type MissingType in MissingType & Type2 was not found.
example/warnings.js:11:11

Keeping Types In Separate File

If the types are numerous and it is desired to put them in a separate JS file (like types.d.ts but for JSDoc) and then import them in code from there for expansions of function's configs, it is possible with the -t argument pointing to the location of XML files. Keeping all files in a types.js file allows to import them from anywhere in the code, or other packages (the file needs to be added to the files field of package.json, if such field exists).

For example, we can create a types.js file with the typal marker:

// types.js
export {} // important for enabling of importing
/* typal types/index.xml closure noSuppress */

The types can be placed in there with typal types.js command. We also add the noSuppress command because the file will not be imported and checked by the Google Closure Compiler therefore the @suppress annotations would be redundant. Now the aim is to update the source code which has a variable of a particular type that we want to expand and we run typal src/index.js -t types/index.xml to do that:

// src/index.js
/**
 * @param {_ns.Config} config
 */
function example(config = {}) {
  const { test } = config
}

// manually add the namespace and dependencies' imports
/**
 * @suppress {nonStandardJsDocs}
 * @typedef {import('stream').Readable} stream.Readable
 */
/**
 * @suppress {nonStandardJsDocs}
 * @typedef {import('../types').Config} _ns.Config
 */

// src/index.js
/**
 * @param {_ns.Config} config The config for the program
 * @param {string} config.test The test property.
 * @param {stream.Readable} config.rs The stream to read.
 */
function example(config = {}) {
  const { test } = config
}

// manually add the namespace and dependencies' imports
/**
 * @suppress {nonStandardJsDocs}
 * @typedef {import('stream').Readable} stream.Readable
 */
/**
 * @suppress {nonStandardJsDocs}
 * @typedef {import('../types').Config} _ns.Config
 */

Any external types referenced in properties must be manually imported, because otherwise their types will be unknown in the scope of the file. This can be done with the snippet that can be put either in the workspace directory as .vscode/import.code-snippets, or configured to be included in User Snippets (⌘⇧ P > Preferences: Configure User Snippets).

{
	"Import Type And Suppress": {
		"prefix": "@typedef",
		"body": [
			"/**",
			" * @suppress {nonStandardJsDocs}",
			" * @typedef {import('$1')$2} $3",
			" */"
		],
		"description": "Insert import typedef"
	}
}

In future, we plan to introduce full-scale management of types so that all import statements will be added automatically by Typal.

Migration

When there are JSDoc types written in JavaScript files, and they need to be put in the types.xml file, it can be done automatically with the --migrate command. In this case, Typal will scan the source code for the type definitions and their properties, defined as @prop or @property tags, and place them either in the output file when specified, or print to the stdout. This will help to move all types into XML declarations, which can then be manually adjusted if necessary, and embedded into the source code using the /* typal types.xml */ marker, and in README documentation using Documentary.

Using Migrate Command
/** * @typedef {import('koa-multer').StorageEngine} StorageEngine * @typedef {import('http').IncomingMessage} IncomingMessage * @typedef {import('koa-multer').File} File */ /** * @typedef {Object} Example An example type. * @typedef {Object} SessionConfig Description of Session Config. * @prop {string} key The cookie key. * @prop {number|'session'} [maxAge=86400000] maxAge in ms. Default is 1 day. * @prop {boolean} [overwrite] Can overwrite or not. Default `true`. * @prop {boolean} [httpOnly] httpOnly or not or not. Default `true`. * @prop {boolean} [signed=false] Signed or not. Default `false`. * @prop {boolean} [rolling] Force a session identifier cookie to be set. * @prop {boolean} [renew] Renew session when session is nearly expired. */
For example, the types above can be extracted into the types file using the typal src/index.js -m [-o types/index.xml] command.
<types> <import name="StorageEngine" from="koa-multer" /> <import name="IncomingMessage" from="http" /> <import name="File" from="koa-multer" /> <type name="Example" desc="An example type." /> <type name="SessionConfig" desc="Description of Session Config."> <prop string name="key"> The cookie key. </prop> <prop type="number|'session'" name="maxAge" default="86400000"> maxAge in ms. Default is 1 day. </prop> <prop boolean name="overwrite" default="true"> Can overwrite or not. </prop> <prop boolean name="httpOnly" default="true"> httpOnly or not or not. </prop> <prop boolean name="signed" default="false"> Signed or not. </prop> <prop opt boolean name="rolling"> Force a session identifier cookie to be set. </prop> <prop opt boolean name="renew"> Renew session when session is nearly expired. </prop> </type> </types>

Schema

The XML schema supports types, imports, methods, properties and functions (which are aliases to properties with special attributes used to construct a function type).

📝 Typal Schema

<types>
  <import from="http" name="IncomingMessage"
    link="https://nodejs.org/api/http.html#incoming_message"
    desc="The readable stream from the connection." />

  <type name="Example" >
    <prop type="string" name="test">The property.</prop>
    <fn async args="number" return="boolean">A method property.</fn>
  </type>

  <method name="sponsor" >
    <arg name="amount">The amount to contribute.</arg>
    Become a sponsor on GitHub.
  </type>
</types>

Markdown Documentation

Typal allows to paste types into documentation using the Documentary package. It will also link the types it knows about for easier navigation. The supported types are based on the Google Closure Compiler types and include the following:

Type: A type which can be linked.

Example: An example type which can link to other types.

Name	Type & Description
type	?Type
The type itself, possibly nullable.
union	!(Type | string)
The union of types.
record	{ t: Type, r }
The record with a type.
application	Object<string, Type>
The application with a type.
function	(this: Type, arg0: string, arg1: !Type) => Type
The function with arguments and return type.
variable-args	(...args: Type[]) => ?
Functions with ... for variable argument types.
vscode-function	(type: Type) => Type
Linking in the VSCode (TypeScript) functions is also supported when properties are written as functions. type* Type: The type.

Used source types.

Copyright & License

GNU Affero General Public License v3.0

	© Art Deco™ 2020